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THE EFFECTS OF TV FORMATS ON MENTAL SKILLS 

by 

Gavriel Salomon and Akiba A, Cohen 
The Hebrew University of Jerusalem 

Abstract 

It was long assumed that exposure to the visual media somehow 
enhances the development of cognitive skills. A theory was developed 
by Dr. G. Salomon which outlined hypothetical ways through which media 
codes could develop particular mental skills. In a series of 
experiments designed to test these hypotheses it was foiind that: 
(a) particular filmic codes explicitly supplant mental skills, thus 
allowing for thfeir im.;.tation and internalization, while others 
develop skills by calling upon them; (b) individuals with poor 
initial mastery of the needed skills tend to improve their mastery 
when skills are supplanted whereas those with good initial mastery 
improve where skills are called upon. 

A later study examined the effects of exposure to Sesame Street 
on the mastery of specific cognitive skills. This field study 
further supported the previous finding. It suggested that psycho- 
logically demanding TV programs, when cross-culturally transferred, 
may gradually enhance universally-shared TV-literacy skills. Ai^other 
result of the study was a battery of TV-related literacy tests. 
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Recently an experiment was carried out by Drs. Salomon & Cohen 
and others r in which the relationships between basic TV and film 
codes to particular literacy rtental skills were examined. Differen't 
versions of the same film, each capitalizing on a different code, 
were produced and shown to fifth graders. It was found that, as 
expected, each code calls upon different mental skills. Children 
with initially better mastery of the relevant skills (i.e. have 
better literacy) - extract more knowledge and meaning from the 
presentation. Where filmic codes supplant a critical skill, rather 
than call upon itv those with initially poor mastery benefit as much 
as those with better mastery. 

A cross-cultural comparison between the U.S. and Israel, in 
which long lasting effects of TV exposure on cognition are studied, 
is presently underway. 
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Our western culture is a highly visual one. Wherever we turn, 

we encounter visual images. Some are more .conventional , while others, 

partucularly television, are much more novel. Added up, these visual' 
stimuli appear to engulf and bombard us constantly 

It seems reasonable to argue that we are apparently quite 
skillful in handling this large quantity of visual material. We must, 
in all likelihood, be equipped with those mental skills which enable 
us to process some, if not all, the information transmitted to us 
through these visual images. 

Or are we? Indeed, what do the visual images require in terms 
of perception, selection and processing of information? Perhaps an 
even more interesting question is what do they do to us, the 
decoders of visual messages? Or do we remain unaffected by them? 

The latter question entails numerous variations. Visual images 
come in many forms and carry many kinds of contents. In addition, 
they are carried by means of different technologies of transmission. 
Clearly then, visual images may affect us, if they do at all, 
through their contents, through their technologies, or through their 
very symbolic nature. However, whereas much thinking and research 
has bean devoted to questions of content and technology, very little 
attention has been paid to the question of how, if at all, do the 
symbolic modes of visual media affect our cognitions. 

It was long assumed that the visual media affect us cognitively 
through their symbolic modes, or rather, their formats of 



O 

ERLC • 



- 4 - 

representation. Carpenter (1960) and later Mcluhan (1965) drew our 
attention to this possibility. Research on language acquisition, 
the well known Whorfian hypothesis, as well as daily observations 

all lent support to the contention that media formats, rather than 

content, affect cognitions. And yet, little empirical evidence can 

be found either to substantiate or to refute this contention* It is 

to this area that our research was devoted. 

It is rather difficult to define the construct of symbolic 
codes, or formats or representation. Let us, therefore, say that 
the codes, or formats, are the sign vehicles, that is - those 
optional components of a communicational message into which a raw 
idea is dressed (Harrison, 1974) . The spatial arrangement and signs 
in a map, the non-notational two-dimensional attributes of drawings, 
the close-up, split-screen, slow motion or time- compress ion in films, 
and the mosaic-like nature as well as the interruptions caused by 
comercials on TV, are fair examples of what is meant by the formats 
of representation. Clearly, there is much overlap among the 
different visual media. But there are also unique additions 
contributed by what each technology potentially makes possible and 
by how producers and artists succeed in mate.^ializing these 
potentialities. Indeed, new formats of representation, such as 
holographs, are still in the process of being developed. 

The theoretical possibility that such representation formats 
can affect cognitions has been explicated elsewhere by Salomon (1972). 
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Briefly stated, it was hypothesized that some representation formats 
can serve a dual function: they serve in an overt communicational 
function, and they may serve in a covert mental representational 
one. Accordingly, it was hypothesized that some representational 
formats, which are used in communication, are internalized to serve 
in a covert function as mental skills . Thus, for instance, a 
cartographer would be expected to internalize the spatial trans- 
formations used as formats in maps, thereby making him better able 
to represent space to himself cognitively, beyond any geographic 
content. Similarly, a child who is a heavy . television consumer 
should be better able to mentally perform specific transformations 
which he has adopted from television formats. 

Interestingly enough, Miinsterberg pointed out as far back as 
1916, that "the photoplay obeys the laws of the mind rather than 
those of the outer world". In other words, the world of film 
externalizes our mental skills. This may be the case. However, for 
the child who is still in the process of acquiring the necessary 
skills to handle film, the formats used by the medium serve as 
external simulators of mental-skills-to-be-learned. He can thus 

imitate, internalize and generalize them co be used as schematic 
2 

mental tools. 

2) Following the publication of this hypothesis, at least two 
similar ones saw light: Lesser (1974) extended the idea into 
the area of television while Tikhomirov (1974) did the same 
with respect to computers. 
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How do representation formats, which are used in conununication, 
affect cognition? And how are their cognitive effects related to 
media literacy? Any message entails ^both content and format. As 
Olson (1974) describes, the content is related to knowledge to be 
acquired, whereas the format is related to the mental skills utilized 
IB the process of acquiring the knowledge. The means of presenting 
information, it is claimed, determine the mental activities by means 
of which knowledge is acquired. Thus, the mental activity brought 
to bear upon the information biases the knowledge acquired. As a 
consequi^nce, any handling of coded messages has two facets: The 
activation of mental skills relevant to the psychological requirements 
of the representation format used in the message, and the acquisition 
of the knowledge entailed in the message. 

The coded message is never a "raw" message; the code used in 
it necessarily biases the information by imposing specific 
modifications on it. A television presentation constructed as a 
kaleidoscope of events differs from a smoothly developing presentation, 
in terms of the modifications each format imposes on the message ; 
even if content is the same. The two messages share some basic 
common formats, and .hence call upon similar mental skills. However, 
to the extent chat they differ, also different mental skills are 
required for the extraction of information from each message. These 
mental skills are needed to overcome the modifications produced by 
the codes . Since each code imposes different modifications. 
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different skills are needed to overcome them. Thus, e.g. in the 
face of close-ups one has to relate the enlarged detail to preceding 
or subsequent long-shots, and m the face of two-dimensionality one 
has to mentally add the third dimension. 

Media literacy is thus one's ability to overcome the 
modifications imposed by the codes of a medium on its messages . 
It can be hypothesized that by overcoming such modifications, a 
person is better able to extract the information from the message. 

We have recently completed an experiment, which was disigned 
to test this hypothesis. Five versions of the same short television 
film were produced. Whereas content was held constant, the dominant 
formats were varied. One version emphasized close-ups, a second 
version was heavily loaded with zooms, a third emphasized logical 
gaps, the fourth - spatial gaps, and the fifth emphasized no 
particular format at all. The subjects, all fifth graders, were 
mxtially pretested on a number of mental-skill tests. Each test 
was assumed to tap mastery of a mental skill which is necessary in 
order to overcome the modifications imposed by a particular 
television code. Later, each group of subjects was exposed to one 
of the filmed versions. Following exposure, the children were given 
a posttest which measured their knowldege of the film's content, 
logic, spatial arrangement and the like. 

As expected, the acquisition of knowledge from each version 
correlated with previous mastery of at least one mental skill. 
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Moreover, these correlation patterns differed for each version. 
Hence, e.g. whereas the skill of relating part to wholes was relevant 
for the acquisition of knowledge in the close-up version, the skill 
of bridging logical gaps was relevant in the logical-gaps versions • 
Similarly, the verbal-logical skill was necessary to overcome the 
modifications of the spacially-f ragmen ted version, but was irrelevant 
in all other versions. 

Two conclusions can be drawn from this experiment. First, 
different codes call upon different mental skills; and second, the 
acquisition of knowledge is strongly facilitated by the activation 
of that skill which aids in overcoming the dominant code of the 
message. 

These findings do not show as yet how media literacy develops, 
nor how, if at all, media affect cognition. They do show, however, 
that cognitive skills are called upon by different codes in the 
service of acquiring knowledge from messages. 

Media literacy may well develop from what seems to be the 
initial starting point, namely - the child's ability to distinguish 
between referent and representation. It is only later on that the 
media-specific skills develop. This development depends, however, 
on at least two factors: The child's experience with the codes, 
and his general cognitive development. 

Our investigation into the effects of Sesame Street on Israeli 
kindergarten and school children provided the opportunity to look 
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into these factors (Salomon et al., 1976). For Israeli children, 
there was much novelty m the program's formats, thus potentially 
causing a mismatch between their already mastered skills and the 
psychological requirements of the novel formats. It should be noted 
that the content of the program was of a rather common and universal 
nature. 

Indeed, accumulated exposure to the program resulted in 
improved mastery of those mental skills which we have assumed to 
be needed to overcome the modifications caused by the novel formats. 
Thus, with experience, mastery improved. Furthermore, this r[iastery 
transferred to other, non-television material . In addition, improved 
skill mastery resulted in better ability to acquire knowledge froia 
other filmed material, indicating that indeed generalized skills 
were cultivated ^ 

However, such cognitive changes were not uniform across ages. 
Whereas second and third graders showed large changes, preschoolers 
did not. The most plausible explanation is that the preschoolers 
did not yet experience the mismatch between their skill mastery and 
the program's requirements. They were equipped to acquire the 
knowledge with which they felt comfortable. For the somewhat older 
children this level of knowledge was not enough and hence their need 
to accommodate their skills to the program's requirements. The 
conclusion we can thus draw from the study is that experience with 
demanding codes is necessary, but that sufficient cognitive 
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development is needed just as much. No wonder, therefore that 
mastery of map reading skills comes much later than film literacy 
(Snyder and Feldman, 1975). 

The evidence thus far supports the contention that media codes 

affect the mastery of relevant mental skills, that these skills 

generalize beyond the handling of specific codes, and that experience 

with the codes as well as cognitive development play crucial roles 
in this process. But how do these cognitive effects occur and what 

IS their nature? 

Another series of experiments which we carried out recently 
(Salomon, 1974) subsequently followed by a study by Rovett (1974) 
examined the process through which media codes affect cognitive 

skills. These studies dealt with filmic codes such as the zoom, 
laying-out of objects, and rotations in space. The unique attribute 
of these codes is that they have the potential of overtly supplanting 
mental transformations, which otherwise would have to be executed 
covertly. Thus, e.g. the zoom supplants the process of relating 
wholes to parts and the process of laying-out solid objects 
supplants the relationship between an object and its 2-D plan. 

The major and consistent findings of the experiments were that 
whenever such codes are used, and hence - specific transformations 
are overtly supplanted, observers imitate and internalize the codes. 
After numerous such exposures, the imitated codes become somewhat 
generalized and can be applied as mantal skills to new stimuli. 
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Thus, the dual function of codes (overt communication and covert 
representation) is realized. 

It is important, however, to notice that not all observers 
go 'through this process of imitation and internalization of the 
codes. Only observers with initially poor mastery of the relevant 
skills tend to imitate them. Those with relatively ^ood mastery 
of the skills do not improve as a result of exposure to supplanting 
codes. Rather, what affects their mastery of mental skills are 
codes which call upon or arouse the mental skills. Thus', e.g. the 
zoom, a supplanting code, improves the skill of relating parts to 
wholes of initially less skillful observers. The close-up, on the 
other hand, which calls upon the same skill, improves mastery of 
the more skillful observer. 

Such were the findings in experiments carried out in laboratory 
settings. They showed how media codes can be made to affect 
cognitive skills. But is this the way media codes influence mental 
skills under more natural conditions? The five-version experiment ^ 
mentioned above, sheds some light on this question. Children's 
ability to relate parts to wholes was strongly related to their 
knowledge acquisition (r = .67) when exposed to the close-up version. 
However, in the zoom version, which supplanted rather than called 
upon that skill, the relationship was substantially weaker (r = .27). 
In other words, when the skill is supplanted by a code, rather than 
called upon, its initial mastery is of much less relevance for the 
extraction of information. 
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The Sesame Street study provides even more direct evidence: 
whenever there were codes which called upon existing skills, only 
already skillful children showed improvements; whenever there were 
supplanting codes, unskillful children benefitted. Thus, codes 
affect cognitions through supplanting them or by calling upon them 
also under natural conditions of exposure. 

We may now offer, by way of a brief summary, an overall view 
of how the media affect cognitive skills and media literacy. Media 
literacy is one's skill in overcoming the modifications imposed on 
messages by codes. This is done in the service of extracting 
knowledge from the coded messages. While engaged in the activity 
of acquiring knowledge from a coded message, the menta'l skills one 
activates to overcome the codes' modifications undergo changes. 
These skills gradually develop, and possibly transfer to other 
domains. However, larger changes take place when a mismatch is 
experienced between available skill mastery and the (novel) 
requirements of a code. 

Two processes possibly account for the acquisition of media 
literacy and for the cognitive changes ^caused by media codes. Some 
codes (particularly of film and television) overtly supplant mental 
processes. They are imitated and internalized by observers, notably 
those who lack mastery of the skills needed to "break" the codes. 
Other codes cannot supplant skills but rather call uprn them. By 
repeatedly doing so", they gradually shape and cultivate the skills. 
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It can be hypothesized that imitation of supplanting codes precedes, 
and that the cultivation of skills by calling upon them follows. But 
this still needs to be empirically tested. 

What our research teaches us thus far, among other things, is 
that media can be made to have desirable cognitive effects, thus ^ 
making it possible to utilize them educationally for novel and 
desirable purposes. Our research also tells us that media seem to 
have cognitive effects under natural conditions of exposure . Whether 
this is indeed a significant cultural factor in a child's course of 
mental development, is a question which we are presently studying 
in a cross-cultural setting comparing American and Israeli /children. 
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